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Research has shown that school leaders can substantially 
influence student outcomes, including student achievement, and 
that some school leaders improve student outcomes at a much 
faster rate than others do.' The strong influence of school leaders 
on student outcomes has led to the creation of student out- 
come-—focused professional development programs for current 
and aspiring school leaders. Balanced Leadership® is one such 
professional development program. Current and aspiring school 
leaders participate in professional development sessions with 
trained facilitators over one or two years, practice what they 
learn between sessions, and can receive additional coaching and 
online support. McREL International, the company that devel- 
oped the Balanced Leadership® program, based the framework 


and content of the professional development on research 
identifying key actions and behaviors of school leaders that are 
associated with improved student outcomes.”# 


This What Works Clearinghouse (WWC) report, part of the 
WWC’s School Leadership topic area, explores the effects of 
Balanced Leadership® on student, teacher, and school leader 
outcomes. The WWC identified three studies of Balanced 
Leadership®. One of these studies meets WWC standards. The 
evidence presented in this report is from one study of the impact 
of Balanced Leadership® on school principals in schools with 

a range of grades—from kindergarten to grade 12—in rural 
school districts in the state of Michigan. 


What Happens When School Leaders Participate in Balanced Leadership®?* 


The evidence indicates that implementing 
Balanced Leadership®: 


¢ May increase school leader retention at the school 


Findings on Balanced Leadership® from the one study that 
met WWC standards are shown in Table 1. For the one out- 
come in this study that met WWC standards, the table reports 


on the performance of the Balanced Leadership® intervention 
group relative to the comparison group and the number of 
studies and principals that contributed to the finding. 


Table 1. Summary of findings on Balanced Leadership® from the one study that meets WWC Standards 


Average performance Evidence meeting WWC standards 


(study findings) (version 4.0) 
Intervention Comparison Number of Number of 
Outcome Effectiveness rating group group studies principals 
School leader retention at the school Potentially positive effects 76% retained 60% retained 1 124 


Note: The average performance (study findings) values are based on an analysis of one outcome measure that meets WWC standards, reported by Jacob et al. (2015). The 
effects of Balanced Leadership® are not known for other outcomes within the School Leadership topic area, including student achievement, student progress in school, social- 
emotional learning and behavior, instructional practice, leadership practice, teacher retention, or measures of school leader retention in the district, state, or teaching profession. 


BOX 1. HOW THE WWC REVIEWS AND DESCRIBES EVIDENCE 


The WWC evaluates evidence based on the quality and results of reviewed studies. The criteria the WWC uses for evaluating 
evidence are defined in the Procedures and Standards Handbooks and the Review Protocols. The studies summarized in this report 
were reviewed under WWC Standards (version 4.0) and the School Leadership topic area protocol (version 4.0). 


To determine the effectiveness rating, the WWC considers what methods each study used, the direction of the effects, and the 
number of studies that tested the intervention. The higher the effectiveness rating, the more certain the WWC is about the reported 
results and about what will happen if the same intervention is implemented again. The following key explains the relationship between 
effectiveness ratings and the statements used in this report: 


Effectiveness Rating Rating interpretation Description of the evidence 


Positive (or negative) effects The intervention is likely to change an 


outcome 


Strong evidence of a positive effect, with no 
overriding contrary evidence 


Potentially positive (or negative) effects The intervention may change an outcome Evidence of a positive effect with no overriding 


contrary evidence 


No discernible effects The intervention may result in little to no No affirmative evidence of effects 


change in an outcome 


Evidence includes studies in at least two of 
these categories: studies with positive effects, 
studies with negative effects, or more studies 
with indeterminate effects than with positive or 
negative effects 


The intervention has inconsistent effects 
onan outcome 


Mixed effects 


How is Balanced Leadership® Implemented? 


The following section provides details of how Balanced 
Leadership® was implemented. This information can 

help educators identify the requirements for implementing 
Balanced Leadership® and determine whether implement- 
ing this intervention would be feasible in their district or 


Comparison group: In the study that contributes to 
this intervention report, principals in the comparison 
group received their school or district’s standard pro- 


fessional development offerings. Several comparison 


school. Information on Balanced Leadership® presented 

in this section comes from the study that meets WWC 
standards (Jacob et al., 2015) and from correspondence with 
the developer. 


Goal: Balanced Leadership® aims to improve student 
outcomes by helping current and aspiring school leaders 
understand and implement a set of actions and behaviors 
of school leaders that may be associated with improved 
student outcomes. 


Target population: The Balanced Leadership® program 
reviewed in this report is intended for principals, assis- 
tant principals, teacher leaders, and others responsible 
for school-level leadership in schools serving kindergarten 
through grade 12. A separate Balanced Leadership® 
program offered by the developer seeks to engage 
district-level leaders. 


Method of delivery: Balanced Leadership® instructional 
specialists provide professional development to groups 

of school leaders during in-person sessions offered at a 
location the school district provides or in a central location 
if multiple districts are participating together (referred to as 
the consortium model). School leaders can receive additional 


group principals described attending a program with 
content similar to Balanced Leadership®. 


one-on-one coaching and online support. In the study that 
contributes to this intervention report, the sessions took 
place in two central locations close to the participating 
school districts. 


Frequency and duration of service: The program is 
customizable, and the number of sessions offered depends 
on what the school or district requests. In the study that 
contributes to this intervention report, principals attended 
10 sessions, each of which was offered over 2 days. 
Though the program is offered over 2 years, the sessions 
in this study took place across parts of 3 school years. The 
study does not describe any additional coaching or online 
support provided by the program. 


Intervention components: Balanced Leadership® includes 
several components, as noted in Table 2. A separate pro- 
gram for district-level leaders, which can be bundled with 
the school-level leader program, is also described below, 
but was not the subject of the study that contributes to this 
intervention report. 


N 


Table 2. Components of Balanced Leadership® 


Key component 


Description 


Framework for actions 
and behaviors of 
school leaders 


On-site professional 
learning for school 
leaders in year one 


On-site professional 
learning for school 
leaders in year two 


The professional development focuses on 21 specific actions and behaviors of school leaders that may be associated with 
improved student outcomes, including designing and implementing school curriculum, instruction, and assessment; fostering 
shared beliefs and community; and monitoring teaching and student learning. 


During the first year, current or aspiring school leaders are offered between one and four two-day on-site professional learning 
sessions focused on the 21 specific actions and behaviors that may be associated with improved student outcomes. The first 
session provides an overview of the program and each of the actions and behaviors of school leaders. The second through 
fourth sessions, in turn, focus on learning how to initiate and lead school change, maintain a focus on school performance, 
and foster shared beliefs and a purposeful school community. The structured training sessions are offered using a case study 
approach, in which participants read about and reflect on real-life applications of the leadership framework. 


For schools that participate in 2 years of the program, sessions in the second year focus on extending and refining the topics 
covered in the first year through group discussions, case studies, and action plans. These activities are offered over two to six 
professional learning sessions, each offered over 2 days. Coaching and additional online supports are available throughout 
the program as requested by the school or district. 


Training for district 


The program can also include professional development for district-level leaders, for example, superintendents or assistant 


staff superintendents, who oversee multiple schools. The district-level training, provided over 1 to 4 days, focuses on setting 
goals, strategies for supporting school leaders, and effective management practices. District staff can also become trained 
facilitators in the Balanced Leadership® program themselves, in order to sustain the program in their districts over time. 


What Does Balanced Leadership® Cost? 


The cost of participating in Balanced Leadership® varies 
depending on the number of school leaders being trained 
and the extent of program customization. The program can 
be implemented using a consortium training model, in which 
school leaders from multiple districts attend the training in 

a central location, to lower program costs to individual 
districts. The preliminary list of cost drivers in this section 

is not designed to be exhaustive; rather, it provides educators 
an overview of the major resources needed to implement 
Balanced Leadership®. Actual program costs are only 
available in consultation with the developer. The information 
described below is based on the information available as of 
July 2019. 


¢ Equipment and materials costs: Participants receive 
activity guides, manuals, and access to online resources as 
part of the contracted cost. 


¢ Personnel costs: McREL provides facilitators to lead 
Balanced Leadership® professional learning sessions and 


For More Information: 


About Balanced Leadership® 
McREL International 
4601 DTC Boulevard: Suite 500 
Denver, CO 80237-2596 


coaching activities. Each participating school leader joins 
up to 10 professional learning sessions over 2 years, and 
additional coaching sessions are available as part of the 
contracted cost. 


Facilities costs: Professional development activities occur 
in a school or administrative building in the school leaders’ 
district. For the consortium training model, the training 
occurs in a centrally located facility provided by one of 
the districts, or a regional education agency. Schools and 
districts are responsible for transportation to and from 
these locations. 


Costs paid by students or parents: None. 
In-kind supports: None. 


Sources of funding: Schools and districts usually cover 
the contracted costs for Balanced Leadership® using 
general funds or any grant funds they may have available. 


Email: info@mcrel.org. Web: www.mcrel.org/balancedleadership/. Phone: (800) 858-6830. 


About the cost of the intervention 


Actual program costs are only available in consultation with the developer. 


Research Summary 


The WWC identified three studies that investigated the effec- 
tiveness of Balanced Leadership® (Figure 1): 


¢ One study meets WWC group design standards without 
reservations 


¢ Two studies are ineligible for review 


The WWC reviews findings on the intervention’s effects on 
eligible outcome domains from studies that meet standards, 
either with or without reservations. Based on this review, the 
WWC generates an effectiveness rating, which summarizes 
how the intervention impacts, or changes, a particular out- 
come domain. Findings from studies that either do not meet 


Figure 1. Effectiveness ratings for Balanced Leadership® 


| study meets WWC 0 studies meet WWC 


jr laverslae lm iaa) 
reservations 


standards without 
reservations 


Contribute to effectiveness ratings 


0 studies do not 


WWC standards or are ineligible for review do not contribute 
to the effectiveness ratings. 


The one study of Balanced Leadership® that meets WWC 
group design standards reported findings on school leader 
retention at the school. No other findings in the study within 
any outcome domain included in the School Leadership topic 
area met WWC group design standards.° The citation for 

the study reviewed for this report is listed in the References 
section on page 8. Citations for the two studies that are inel- 
igible for review and the reasons the WWC determined they 
were ineligible are also listed in the References section. 


2 studies are 


meet WWC 
standards 


ineligible for 
review 


Do not contribute to effectiveness ratings 


Balanced Leadership® has potentially positive effects on school leader retention at the school 


The WWC determined that one study that meets WWC group design standards without reservations shows evidence of 


a positive effect of Balanced Leadership® on school leader retention at the school with no overriding contrary evidence 
(Jacob et al., 2015). 


Main Findings 


® 


Table 3 shows the findings from one Balanced Leadership rating for school leader retention at the school is potentially 


study that meets WWC standards. The table includes WWC positive effects, indicating evidence of a positive effect with 
calculations of the mean difference, effect size, and perfor- no overriding contrary evidence. This finding is based on 
mance of the intervention group relative to the comparison 124 school principals. 


group. Based on findings from this study, the effectiveness 


Table 3. Findings by outcome domain from the one study of Balanced Leadership® that meets WWC Standards 


Mean 
(standard deviation) WWC calculations 
Sample Intervention Comparison Mean Effect Improvement 
Measure (study) Study sample size group group difference __size index p-value 
Principal turnover School principals 124 24% 40% -16 0.45 +17 10 
at the school 
after 3 years 
(Jacob et al., 2015)? 
Outcome average for school leader retention at the school 0.45 +17 


Notes: For principal turnover, lower mean scores indicate more favorable results; the WWC-calculated findings on school leader retention in Table 1 and the effect size and 
improvement index reported in this table are adjusted so that a positive number favors the intervention group. The effect size is a standardized measure of the effect of an 
intervention on outcomes, representing the average change expected for all individuals who are given the intervention (measured in standard deviations of the outcome 
measure). An indicator of the effect of the intervention, the improvement index is a standardized measure of the relative performance of the intervention group to the comparison 
group in percentile terms. A positive number favors the intervention group. For example, an improvement index of +17 means that the expected percentile rank of the average 
comparison group principal would increase by 17 points if the principal received Balanced Leadership®. A positive improvement index does not necessarily mean the estimated 
effect is statistically significant. 

4 The findings in Jacob et al. (2015) did not require corrections for clustering, multiple comparisons adjustments, or difference-in-differences adjustments. The unadjusted means 
and sample sizes presented here were provided in response to an author query. This study is characterized as having a potentially positive effect on school leader retention at 
the school because there is evidence of a positive effect with no overriding contrary evidence. For more information, please refer to the WWC Procedures Handbook, version 
4.0, page 22. 


In What Context Was Balanced Leadership® Studied? 


The following section provides information on the set- 

ting of the one study of Balanced Leadership® that meets 
WWC standards, and a description of the participants in the 
research. This information can help educators understand 


WHERE THE STUDY WAS CONDUCTED 


the context in which the study of Balanced Leadership® was 
conducted, and determine whether the program might be 
suitable for their setting. 


ee, study, 124 public schools across as many as 74 school districts in northern Michigan 


Student Race 


190% 0% 


White Minority 


Free & Reduced-Price Lunch: 47% 


Grades a 1) (2 } a) 


Details of Each Study that Meets WWC Standards 


This section presents details for the study of Balanced 
Leadership® that meets WWC standards. These details 
include the full study reference, findings description, findings 
summary, and description of study characteristics. A sum- 
mary of domain findings for the study is presented below, 
followed by a description of the study characteristics. These 
study-level details include contextual information around the 
study setting, methods, sample, intervention group, compar- 
ison group, outcomes, and implementation details. For addi- 
tional information, readers should refer to the original study. 


Research details for Jacob et al. (2015) 


Jacob, R., Goddard, R., Kim, M., Miller, R., & Goddard Y. 
(2015). Exploring the causal impact of the McREL Balanced 


Table 4. Summary of findings from Jacob et al. (2015) 


Rural districts 
Principal Race 
uelehZ) 


White 


Principals with Graduate Degrees: 77% 


0000000 0 Of 


Grades K-12 


Leadership program on leadership, principal efficacy, instruc- 
tional climate, educator turnover, and student achievement. 
Educational Evaluation and Policy Analysis, 37(3), 314-332. 
Retrieved from https://eric.ed.gov/?id=EJ1072764. 


Findings from Jacob et al. (2015) show evidence of poten- 
tially positive effects of Balanced Leadership® on school 
leader retention at the school (Table 4). This finding is based 
on an outcome analysis that includes 124 principals. The 
finding and research details summarized for this study come 
from two related citations, including the primary study listed 
above. See the References section on page 8 for a list of all 
related publications. 


Meets WWC Group Design Standards Without Reservations 


Study findings 
Sample Average Improvement Statistically 
Outcome domain size effect size index significant 
School leader retention at the school 124 principals 0.45 +17 No 


Table 5. Description of study characteristics for Jacob et al. (2015) 


WWC evidence 
rating 


Setting 


Methods 


Study sample 


Intervention 
group 


Comparison 
group 


Outcomes and 
measurement 


Additional 
implementation 
details 


Meets WWC Group Design Standards Without Reservations. This is a randomized controlled trial (RCT) with low 
attrition.® For more information on how the WWC assigns study ratings, please see the WWC Procedures and Standards 
Handbooks (version 4.0) and WWC Standards Briefs, available on the WWC website. 


The study includes 124 rural schools spread across as many as 74 school districts in northern Michigan, including 
the Upper Peninsula. Only schools that were public (including charter or magnet) and served grades 3 to 5 inclusively 
(for example, K-5 or K-12) were eligible to participate in the study. 


This study is an RCT that assigned schools and their principals to intervention or comparison conditions. Of 126 schools 
and principals recruited for the study, approximately half were randomly assigned to an intervention group in which 

the principals participated in the Balanced Leadership® program, and half to a “business as usual” comparison group. 
Recruitment was conducted by contacting superintendents who then signed a letter of agreement allowing their principals 
to participate. Principals were then contacted and invited to participate in the study. 


Random assignment was conducted using a stratified approach. Researchers placed the 126 schools into 12 groups based 
on proximity to an urban center (the degree of rurality) and socioeconomic status. Schools within each group were assigned 
a random number and then ordered accordingly from lowest to highest. The first half of the schools in each sorted list was 
assigned to the intervention group; the second half was assigned to the comparison group. After assignment, 62 schools 
were in the Balanced Leadership? intervention group and 64 schools were in the business-as-usual comparison group. 


The sample loss after random assignment (attrition) was within the acceptable threshold for the review: the overall attrition 
rate was 2% and differential attrition was 3 percentage points. A total of 124 schools and principals remained in the analysis 
after 3 years. 


One hundred twenty-four schools participated in the study for all 3 years. This included 124 principals, as many as 
1,764 teachers, and more than 30,000 students. Outcomes using the analytic samples of teachers and students did not 
meet WWC group design standards to be included in this review. 


In study schools, which averaged 300 students each, approximately 47% of students were eligible for the free- or reduced- 
price meal program and 10% were minorities. Forty-three percent of principals were female, 77% had advanced degrees, 
and all were White. No descriptive information was provided for the sample of teachers. 


Principals in the intervention group were offered 10 Balanced Leadership® program sessions, each of which took place 
over 2 days. These sessions were spread across 3 academic years, starting in January 2009 (the 2008-09 school year) 
and ending in October 2010 (the 2010-11 school year). The consortium training model was used, with principals attending 
training together at one of two locations closest to their districts. 


No alternative intervention was designed for the comparison schools. Principals in those schools received their school or 
district's standard professional development offerings. Of the school principals in the comparison group, 12% reported 
attending a program with content similar to the intervention. 


Study authors reported findings on one outcome measure that is eligible for review under the School Leadership topic area and 
met WWC group design standards. This outcome, principal turnover at the school, was reviewed in the school leader retention 
at the school domain. Principal turnover between the baseline year (the 2008-09 school year) and the end of the third year of 
the study (the 2010-11 school year) was assessed using administrative data from the Michigan Department of Education. 


The study also reported findings for teacher turnover, which was reviewed in the teacher retention at the school domain. 
This outcome did not meet WWC group design standards because teachers joining study schools after random assignment 
may have posed a risk of bias, and the authors do not present information to confirm that the analytic sample of teachers in 
the intervention and comparison groups is equivalent at baseline. 


In addition, the study reported findings for a survey of principal leadership and school climate as reported by principals 

and teachers. These findings were not eligible for review in the leadership practice domain. Eligible measures in this 
domain must demonstrate a link between the measure and a student outcome (for example, student achievement), but this 
information was not available.’ 


Finally, the study reported findings for student achievement on math and reading standardized tests in grades 3-5. These 
outcomes did not meet WWC group design standards because the authors do not present information the WWC needed to 
assess the risk of bias due to nonresponse of students. 


The Balanced Leadership® professional development sessions were facilitated by full-time training consultants employed 

by the program developer. Training consultants were required to complete practice training sessions with more experienced 
facilitators before they were allowed to facilitate sessions in the field. No additional information on support for implementation is 
described in the study. 
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Endnotes 


'Several studies describe differences across school leaders in their 
impacts on student achievement, including Dhuey and Smith 
(2018); Branch et al. (2012); and Leithwood et al. (2004). 


? The research that McREL International used to identify the actions 
and behaviors of school leaders associated with improved student 
outcomes includes School Leadership That Works: From Research 
to Results (Marzano et al. 2005). A review of this research by the 
WWC is beyond the scope of this review. 


3 The descriptive information for this intervention comes from Jacob 
et al. (2015) and the intervention developer. The What Works 
Clearinghouse (WWC) requests developers review the interven- 
tion description sections for accuracy from their perspective. The 
WWC provided the developer with the intervention description in 
July 2019 and the WWC incorporated feedback from the develop- 
er. Further verification of the accuracy of the descriptive informa- 
tion for this intervention is beyond the scope of this review. 


‘The literature search reflects documents publicly available by June 
2019. Reviews of the studies in this report used the standards from 
the WWC Procedures and Standards Handbook (version 4.0) and 
the School Leadership review protocol (version 4.0). The evidence 
presented in this report is based on available research. Findings 
and conclusions could change as new research becomes available. 


° Student, teacher, and school leader outcomes can be reviewed 
under the School Leadership topic area protocol. The study that 
met WWC group design standards did not report findings that met 
standards in other outcome domains. For student outcomes, these 
domains are general literacy achievement, general mathematics 
achievement, general science achievement, general social studies 
achievement, general achievement, English language proficien- 
cy, staying in school, progression in school, completing school, 
student social interaction, observed individual behavior, student 
emotional status, and student engagement in school. For teach- 
er outcomes, these domains are instructional practice, teacher 
attendance, teacher retention at the school, teacher retention in the 
school district, teacher retention in the state, and teacher retention 
in the profession. For school leaders, these outcome domains are 
leadership practice, school leader retention in the school district, 
school leader retention in the state, and school leader retention in 
the profession. 


6 This study meets WWC group design standards without reserva- 
tions because one outcome (principal turnover) is from a low-at- 
trition RCT where the risk of bias from individual nonresponse 
is low. Other measures in this study do not meet standards as 
described in Table 5. This review of Jacob et al. (2015) assigned 
the same study rating as the prior review of the study conducted 
for the IES grant competition (based on a previous version of 
the publication released online in 2014 prior to print). This prior 
review was conducted under the version 3.0 standards using the 
Single Study Review Protocol (version 2.0). 


7The survey measure was also reviewed as a supplemental outcome; 
however, no survey findings from the analytic sample of principals 
met WWC group design standards because of high cluster-level 
attrition from survey nonresponse and non-equivalence on several 
baseline measures from within the outcome domain. The analytic 
sample of teachers did not meet WWC group design standards 
because of high cluster-level attrition from survey nonresponse. 
Though the authors present information to demonstrate equiva- 
lence of the clusters at baseline, they do not present information 
the WWC needed to assess the risk of bias due to nonresponse of 
teachers at follow-up. 
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